The problem of image alignment or image registration has been extensively researched and successful methods for solving this problem have been developed. Some of these methods are based on matching extracted local image features. Other approaches are based on directly matching image intensities. However, all these methods share one basic assumption: that there is sufficient overlap between the images to allow extraction of common image properties, namely, that there is sufficient “similarity” between the images. The term similarity is used here in the broadest sense, for example: gray-level similarity, feature similarity, similarity of frequencies and statistical similarity, such as mutual information. Consequently, the prior art does not provide alignment between images where there is very little similarity between the images or where there is no spatial overlap between the images.
However, a sequence of images contains much more information than any individual image does. For example: a sequence of images of a scene contains temporal changes caused by dynamic changes in the scene, or temporal changes induced by the motion of the camera. This information is employed by the present invention to provide alignment between images of the two sequences of images even when the sequences have no spatial overlap or have very low similarity.
State of the art methods for sequence-to-sequence alignment, and other relevant technologies, are described in:    [1] S. Avidan and A. Shashua. Threading fundamental matrices. In European Conference on Computer Vision, 1998.    [2] P. A. Beardsley, P. H. S. Torr and A. Zisserman. 3D model acquisition from extended image sequences. In Proc. 4th European Conference on Computer Vision, LNCS 1065, Cambridge, pages 683-695, 1996.    [3] P. R. Burt and R. J. Kolczynski. Enhanced image capture through fusion. In International Conference on Computer Vision, 1993.    [4] Y. Caspi and M. Irani. A step towards sequence-to-sequence alignment. In IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, S.C., June 2000.    [5] D. Demirdijian, A. Zisserman and R. Horaud. Stereo autocalibration from one plane. In European Conference on Computer Vision, 2000.    [6] Y. Dufournaud, C. Schmid and R. Horaud. Matching images with different resolutions. In IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, S.C., June 2000.    [7] C. E. Pearson (ed). Handbook of applied mathematics—Second Edition. Van Nostrand Reinhold Company, New York, 1983, pp 898.    [8] F. R. Gantmakher. The theory of matrices. Chapter VIII, Chelsea Pub., New York, 1959.    [9] Gene Golub and Charles Van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore and London, pp. 123-127, 1989    [10] Richard I. Hartley. In Defence of the 8-point Algorithm. In Pattern Recognition and Machine Intelligence 19 (6) June pages 580-593 1997    [11] R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Chapter 3, Cambridge university press, Cambridge, 2000.    [12] R. Horaud and G. Csurka. Reconstruction using motions of a stereo rig. In International Conference on Computer Vision, pages 96-103, 1998.    [13] R. Horaud and F. Dornaika. Hand-eye calibration. International Journal of Robotics Research, 14(3):195-210, June 1995.    [14] M. Irani and P. Anandan. About direct methods. In Vision Algorithms Workshop, pages 267-277, Corfu, 1999.    [15] M. Irani, P. Anandan and D. Weinshall. From reference frames to reference planes: Multi-view parallax geometry and applications. In European Conference on Computer Vision, Freiburg, June 1998.    [16] M. Irani, B. Rousso and S. Peleg. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5-16, January 1994.
[17] R. Kumar, P. Anandan and K. Hanna. Direct recovery of shape from multiple views: parallax based approach. In International Conference on Pattern Recognition, 1994.    [18] Harpreet Sawhney. 3D geometry from planar parallax. In IEEE Conference on Computer Vision and Pattern Recognition, June 1994.    [19] A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, pages 483-489, Seattle, Wash., June 1994.    [20] G. P. Stein. Tracking from multiple view points: Self-calibration of space and time. In DARPA IU Workshop, pages 1037-1042, 1998.    [21] P. H. S. Torr and A. Zisserman. Feature based methods for structure and motion estimation. In Vision Algorithms Workshop, pages 279-329, Corfu, 1999.    [22] R. Y. Tsai and R. K. Lenz. A new technique for full autonomous and efficient. 3D robotics hand/eye calibration. IEEE Journal of Robotics and Automation, 5(3):345-358, June 1989.    [23] P. Viola and W. Wells III. Alignment by maximization of mutual information. In International Conference on Computer Vision, pages 16-23, 1995.    [24] A. Zisserman, P. A. Beardsley and I. D. Reid. Metric calibration of a stereo rig. In Workshop on Representations of Visual Scenes, pages 93-100, 1995.    [25] P. H. S. Torr and A. Zisserman. Feature based methods for structure and motion estimation. In Vision Algorithms Workshop, pages 279-290, Corfu, 1999.    [26] Z. Zhang, R. Deriche, O. Faugeras, and Q. Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87-119, 1995.
The disclosures of all publications mentioned in the specification and of the publications cited therein are hereby incorporated by reference.